HAVOC: Small-scale histomic mapping of cancer biodiversity across large tissue distances using deep neural networks.

Intratumoral heterogeneity can wreak havoc on current precision medicine strategies because of challenges in sufficient sampling of geographically separated areas of biodiversity distributed across centimeter-scale tumor distances. To address this gap, we developed a deep learning pipeline that leverages histomorphologic fingerprints of tissue to create "Histomic Atlases of Variation Of Cancers" (HAVOC). Using a number of objective molecular readouts, we demonstrate that HAVOC can define regional cancer boundaries with distinct biology. Using larger tumor specimens, we show that HAVOC can map biodiversity even across multiple tissue sections. By guiding profiling of 19 partitions across six high-grade gliomas, HAVOC revealed that distinct differentiation states can often coexist and be regionally distributed within these tumors. Last, to highlight generalizability, we benchmark HAVOC on additional tumor types. Together, we establish HAVOC as a versatile tool to generate small-scale maps of tissue heterogeneity and guide regional deployment of molecular resources to relevant biodiverse niches.

Standardizing analysis of intra-tumoral heterogeneity with computational pathology.

Many malignant cancers like glioblastoma are highly adaptive diseases that dynamically change their regional biology to survive and thrive under diverse microenvironmental and therapeutic pressures. While the concept of intra-tumoral heterogeneity has become a major paradigm in cancer research and care, systematic approaches to assess and document bio-variation in cancer are still in their infancy. Here we discuss existing approaches and challenges to documenting intra-tumoral heterogeneity and emerging computational approaches that leverage artificial intelligence to begin to overcome these limitations. We propose how these emerging techniques can be coupled with a diversity of molecular tools to address intra-tumoral heterogeneity more systematically in research and in practice, especially across larger specimens and longitudinal analyses. Systematic documentation and characterization of heterogeneity across entire tumor specimens and their longitudinal evolution has the potential to improve our understanding and treatment of cancer.

Compound computer vision workflow for efficient and automated immunohistochemical analysis of whole slide images.

AIMS: Immunohistochemistry (IHC) assessment of tissue is a central component of the modern pathology workflow, but quantification is challenged by subjective estimates by pathologists or manual steps in semi-automated digital tools. This study integrates various computer vision tools to develop a fully automated workflow for quantifying Ki-67, a standard IHC test used to assess cell proliferation on digital whole slide images (WSIs). METHODS: We create an automated nuclear segmentation strategy by deploying a Mask R-CNN classifier to recognise and count 3,3'-diaminobenzidine positive and negative nuclei. To further improve automation, we replaced manual selection of regions of interest (ROIs) by aligning Ki-67 WSIs with corresponding H&E-stained sections, using scale-invariant feature transform (SIFT) and a conventional histomorphological convolutional neural networks to define tumour-rich areas for quantification. RESULTS: The Mask R-CNN was tested on 147 images generated from 34 brain tumour Ki-67 WSIs and showed a high concordance with aggregate pathologists' estimates ([Formula: see text] assessors; [Formula: see text] r=0.9750). Concordance of each assessor's Ki-67 estimates was higher when compared with the Mask R-CNN than between individual assessors (ravg=0.9322 vs 0.8703; p=0.0213). Coupling the Mask R-CNN with SIFT-CNN workflow demonstrated ROIs can be automatically chosen and partially sampled to improve automation and dramatically decrease computational time (average: 88.55-19.28 min; p<0.0001). CONCLUSIONS: We show how innovations in computer vision can be serially compounded to automate and improve implementation in clinical workflows. Generalisation of this approach to other ancillary studies has significant implications for computational pathology.

The Brain Protein Atlas: A conglomerate of proteomics datasets of human neural tissue.

The human brain represents one of the most complex biological structures with significant spatiotemporal molecular plasticity occurring through early development, learning, aging, and disease. While much progress has been made in mapping its transcriptional architecture, more downstream phenotypic readouts are relatively scarce due to limitations with tissue heterogeneity and accessibility, as well as an inability to amplify protein species prior to global -OMICS analysis. To address some of these barriers, our group has recently focused on using mass-spectrometry workflows compatible with small amounts of formalin-fixed paraffin-embedded tissue samples. This has enabled exploration into spatiotemporal proteomic signatures of the brain and disease across otherwise inaccessible neurodevelopmental timepoints and anatomical niches. Given the similar theme and approaches, we introduce an integrated online portal, "The Brain Protein Atlas (BPA)" (www.brainproteinatlas.org), representing a public resource that allows users to access and explore these amalgamated datasets. Specifically, this portal contains a growing set of peer-reviewed mass-spectrometry-based proteomic datasets, including spatiotemporal profiles of human cerebral development, diffuse gliomas, clinically aggressive meningiomas, and a detailed anatomic atlas of glioblastoma. One barrier to entry in mass spectrometry-based proteomics data analysis is the steep learning curve required to extract biologically relevant data. BPA, therefore, includes several built-in analytical tools to generate relevant plots (e.g., volcano plots, heatmaps, boxplots, and scatter plots) and evaluate the spatiotemporal patterns of proteins of interest. Future iterations aim to expand available datasets, including those generated by the community at large, and analytical tools for exploration. Ultimately, BPA aims to improve knowledge dissemination of proteomic information across the neuroscience community in hopes of accelerating the biological understanding of the brain and various maladies.

Integrating morphologic and molecular histopathological features through whole slide image registration and deep learning.

BACKGROUND: Modern molecular pathology workflows in neuro-oncology heavily rely on the integration of morphologic and immunohistochemical patterns for analysis, classification, and prognostication. However, despite the recent emergence of digital pathology platforms and artificial intelligence-driven computational image analysis tools, automating the integration of histomorphologic information found across these multiple studies is challenged by large files sizes of whole slide images (WSIs) and shifts/rotations in tissue sections introduced during slide preparation. METHODS: To address this, we develop a workflow that couples different computer vision tools including scale-invariant feature transform (SIFT) and deep learning to efficiently align and integrate histopathological information found across multiple independent studies. We highlight the utility and automation potential of this workflow in the molecular subclassification and discovery of previously unappreciated spatial patterns in diffuse gliomas. RESULTS: First, we show how a SIFT-driven computer vision workflow was effective at automated WSI alignment in a cohort of 107 randomly selected surgical neuropathology cases (97/107 (91%) showing appropriate matches, AUC = 0.96). This alignment allows our AI-driven diagnostic workflow to not only differentiate different brain tumor types, but also integrate and carry out molecular subclassification of diffuse gliomas using relevant immunohistochemical biomarkers (IDH1-R132H, ATRX). To highlight the discovery potential of this workflow, we also examined spatial distributions of tumors showing heterogenous expression of the proliferation marker MIB1 and Olig2. This analysis helped uncover an interesting and unappreciated association of Olig2 positive and proliferative areas in some gliomas (r = 0.62). CONCLUSION: This efficient neuropathologist-inspired workflow provides a generalizable approach to help automate a variety of advanced immunohistochemically compatible diagnostic and discovery exercises in surgical neuropathology and neuro-oncology.

Topographic mapping of the glioblastoma proteome reveals a triple-axis model of intra-tumoral heterogeneity.

Glioblastoma is an aggressive form of brain cancer with well-established patterns of intra-tumoral heterogeneity implicated in treatment resistance and progression. While regional and single cell transcriptomic variations of glioblastoma have been recently resolved, downstream phenotype-level proteomic programs have yet to be assigned across glioblastoma's hallmark histomorphologic niches. Here, we leverage mass spectrometry to spatially align abundance levels of 4,794 proteins to distinct histologic patterns across 20 patients and propose diverse molecular programs operational within these regional tumor compartments. Using machine learning, we overlay concordant transcriptional information, and define two distinct proteogenomic programs, MYC- and KRAS-axis hereon, that cooperate with hypoxia to produce a tri-dimensional model of intra-tumoral heterogeneity. Moreover, we highlight differential drug sensitivities and relative chemoresistance in glioblastoma cell lines with enhanced KRAS programs. Importantly, these pharmacological differences are less pronounced in transcriptional glioblastoma subgroups suggesting that this model may provide insights for targeting heterogeneity and overcoming therapy resistance.

Unsupervised Resolution of Histomorphologic Heterogeneity in Renal Cell Carcinoma Using a Brain Tumor-Educated Neural Network.

PURPOSE: Applications of deep learning to histopathology have proven capable of expert-level performance, but approaches have largely focused on supervised classification tasks requiring context-specific training and deployment. More generalizable workflows that can be easily shared across subspecialties could help accelerate and broaden adoption. Here, we hypothesized that histology-optimized feature representations, generated by a convolutional neural network (CNN) during supervised learning, are transferable and can resolve meaningful differences in large-scale, discovery-type unsupervised analyses. METHODS: We used a CNN, previously trained to recognize brain tumor histomorphologies, to extract 512 feature representations from > 550 digital whole-slide images (WSIs) of renal cell carcinomas (RCCs) from The Cancer Genome Atlas and other previously unencountered tumors. We use these extracted feature vectors to conduct unsupervised image-set clustering and analyze the clinical and biologic relevance of the intra- and interpatient subgroups generated. RESULTS: Within individual WSIs, feature-based clustering could reliably segment tumor regions and other relevant histopathologic subpatterns (eg, adenosquamous and poorly differentiated regions). Across the larger RCC cohorts, clustering extracted features generated subgroups enriched for clinically relevant subtypes (eg, papillary RCC) and outcomes (eg, survival). Importantly, individual feature activation mapping highlighted salient subtype-specific patterns and features of malignancies (eg, nuclear grade, sarcomatous change) contributing to subgroupings. Moreover, some proposed clusters were enriched for recurring, human-based RCC-subtype misclassifications. CONCLUSION: Our data support that CNNs, pretrained on large histologic datasets, can extend learned representations to novel scenarios and resolve clinically relevant intra- and interpatient tissue-pattern differences without explicit instruction or additional optimization. Repositioning of existing histology-educated networks could provide scalable approaches for image classification, quality assurance, and discovery of unappreciated patterns and subgroups of disease.

Unsupervised Machine Learning in Pathology: The Next Frontier.

Applications of artificial intelligence and particularly deep learning to aid pathologists in carrying out laborious and qualitative tasks in histopathologic image analysis have now become ubiquitous. We introduce and illustrate how unsupervised machine learning workflows can be deployed in existing pathology workflows to begin learning autonomously through exploration and without the need for extensive direction. Although still in its infancy, this type of machine learning, which more closely mirrors human intelligence, stands to add another exciting layer of innovation to computational pathology and accelerate the transition to autonomous pathologic tissue analysis.

Physician perspectives on integration of artificial intelligence into diagnostic pathology.

Advancements in computer vision and artificial intelligence (AI) carry the potential to make significant contributions to health care, particularly in diagnostic specialties such as radiology and pathology. The impact of these technologies on physician stakeholders is the subject of significant speculation. There is however a dearth of information regarding the opinions, enthusiasm, and concerns of the pathology community at large. Here, we report results from a survey of 487 pathologist-respondents practicing in 54 countries, conducted to examine perspectives on AI implementation in clinical practice. Despite limitations, including difficulty with quantifying response bias and verifying identity of respondents to this anonymous and voluntary survey, several interesting findings were uncovered. Overall, respondents carried generally positive attitudes towards AI, with nearly 75% reporting interest or excitement in AI as a diagnostic tool to facilitate improvements in workflow efficiency and quality assurance in pathology. Importantly, even within the more optimistic cohort, a significant number of respondents endorsed concerns about AI, including the potential for job displacement and replacement. Overall, around 80% of respondents predicted the introduction of AI technology in the pathology laboratory within the coming decade. Attempts to identify statistically significant demographic characteristics (e.g., age, sex, type/place of practice) predictive of attitudes towards AI using Kolmogorov-Smirnov (KS) testing revealed several associations. Important themes which were commented on by respondents included the need for increasing efforts towards physician training and resolving medical-legal implications prior to the generalized implementation of AI in pathology.

Deep learning for image analysis: Personalizing medicine closer to the point of care.

The precision-based revolution in medicine continues to demand stratification of patients into smaller and more personalized subgroups. While genomic technologies have largely led this movement, diagnostic results can take days to weeks to generate. Management at, or closer to, the point of care still heavily relies on the subjective qualitative interpretation of clinical and diagnostic imaging findings. New and emerging technological advances in artificial intelligence (AI) now appear poised to help bring objectivity and precision to these traditionally qualitative analytic tools. In particular, one specific form of AI, known as deep learning, is achieving expert-level disease classifications in many areas of diagnostic medicine dependent on visual and image-based findings. Here, we briefly review concepts of deep learning, and more specifically recent developments in convolutional neural networks (CNNs), to highlight their transformative potential in personalized medicine and, in particular, diagnostic histopathology. Understanding the opportunities and challenges of these quantitative machine-based decision support tools is critical to their widespread introduction into routine diagnostics.

Visualizing histopathologic deep learning classification and anomaly detection using nonlinear feature space dimensionality reduction.

BACKGROUND: There is growing interest in utilizing artificial intelligence, and particularly deep learning, for computer vision in histopathology. While accumulating studies highlight expert-level performance of convolutional neural networks (CNNs) on focused classification tasks, most studies rely on probability distribution scores with empirically defined cutoff values based on post-hoc analysis. More generalizable tools that allow humans to visualize histology-based deep learning inferences and decision making are scarce. RESULTS: Here, we leverage t-distributed Stochastic Neighbor Embedding (t-SNE) to reduce dimensionality and depict how CNNs organize histomorphologic information. Unique to our workflow, we develop a quantitative and transparent approach to visualizing classification decisions prior to softmax compression. By discretizing the relationships between classes on the t-SNE plot, we show we can super-impose randomly sampled regions of test images and use their distribution to render statistically-driven classifications. Therefore, in addition to providing intuitive outputs for human review, this visual approach can carry out automated and objective multi-class classifications similar to more traditional and less-transparent categorical probability distribution scores. Importantly, this novel classification approach is driven by a priori statistically defined cutoffs. It therefore serves as a generalizable classification and anomaly detection tool less reliant on post-hoc tuning. CONCLUSION: Routine incorporation of this convenient approach for quantitative visualization and error reduction in histopathology aims to accelerate early adoption of CNNs into generalized real-world applications where unanticipated and previously untrained classes are often encountered.